Skip to content

Audit remediation: Worker security, content, and release gates#6

Merged
adewale merged 16 commits into
mainfrom
claude/quirky-bell-8fqrne
Jul 10, 2026
Merged

Audit remediation: Worker security, content, and release gates#6
adewale merged 16 commits into
mainfrom
claude/quirky-bell-8fqrne

Conversation

@adewale

@adewale adewale commented Jun 12, 2026

Copy link
Copy Markdown
Owner

What

Completes the 2026-06-11 codebase audit: hardens the Dynamic Worker and Turnstile boundary, improves editorial/journey discoverability, and makes generated assets and releases verifiable before merge or deploy.

This is a deliberately cross-cutting, 247-file audit-remediation change (+7,088/−13,531), including regenerated public assets, 29 example-source corrections, CI/hooks, tests, documentation, and review evidence.

Why

The audited implementation could buffer unbounded program output/error traces, return server errors for malformed form or Turnstile-provider failures, let journey registry data drift between local and Worker runtime modes, omit metadata for the sitemap-listed /journeys index, and deploy generated artifacts not represented by the commit. The audit also found gaps in generated-output prevention and reviewer evidence.

How

  • Bounds Dynamic Worker stdout and traceback output to 64 KiB; returns controlled 413 responses; rejects malformed UTF-8 form data before execution; and fails Turnstile transport/status/JSON failures closed.
  • Disables public workers.dev fallback and documents the required custom-domain Cloudflare WAF/rate-limit configuration.
  • Loads journey and edge-label data from TOML in local and embedded-Worker modes; adds journey pages, figures, JSON-LD, social cards, and complete /journeys index metadata.
  • Adds deterministic social-card provenance and exact JPEG-set validation, plus pre-commit and protected-branch verification to reject generated drift before merge.
  • Makes make deploy verify committed generated output, sync the ignored Python Workers dependency bundle, then invoke Wrangler.
  • Adds behavioral regression coverage for runner limits, malformed forms, Turnstile failure modes, registry loading, journey-index metadata, social-card provenance, and inline rendering.

Testing

  • make verify with a local pywrangler dev --port 9696 Worker: 183 tests plus SEO/cache, quality, browser/layout, search, Ruff, social-card, and generated-file checks pass.
  • GitHub Actions verify passes on the final rebased head (075a198).
  • make check-social-cards verifies provenance and the exact 118-card JPEG set.
  • make -n deploy confirms generated-output validation precedes dependency sync and Wrangler deploy.
  • Regression guard: the journey-index metadata test failed before og:image and JSON-LD were implemented, then passed with the tracked public/og/journeys.jpg artifact.
  • Regression guard: removing bounded traceback handling makes test_generated_worker_enforces_utf8_output_cap fail (500 != 413).

Visual evidence

Runner before/after — /examples/values preserves the editor/output layout while changing the Run action from #FF4801 to WCAG-AA #C83800.

Before: runner action color
After: runner action color

Journey page — /journeys/runtime renders its overview, first outcome, figure, and linked examples at a 1200×1260 desktop viewport.

Runtime journey page

Social card — exact tracked 1200×630 JPEG referenced by /journeys through og:image.

Journeys social card

docs/pr-evidence/README.md records source commits, SHA-256 hashes, selectors, and the follow-up TODO for an executable base/head capture wrapper. The wrapper is intentionally reviewer tooling rather than a pixel-comparison CI gate, so browser/font rasterization differences cannot make Verify flaky.

Risk

  • The 64 KiB runner-output cap intentionally rejects oversized output/error traces rather than buffering them.
  • make deploy now stops until generated output is regenerated, reviewed, and committed.
  • Cloudflare zone rate limiting remains an external production prerequisite; the repository cannot declare or verify account-level WAF rules.
  • JPEG bytes vary across platforms; CI verifies deterministic card-input provenance and the exact expected file set rather than raster bytes.
  • CodeMirror/Shiki retain pinned esm.sh imports; vendoring their complete graph is a separate dependency-bundling effort.

@adewale adewale changed the title Audit fixes: content gates, specs, examples, and security headers Audit remediation: quality gates with teeth, pinned toolchain, content and runtime fixes Jun 12, 2026
claude and others added 10 commits July 10, 2026 02:00
Covers code, docs, CI, quality gates, example content, figures, and
tests, with verified findings and improvement suggestions.

https://claude.ai/code/session_018EpcfENTg7herPR712vZeE
- Route every Makefile python target through uv run --python 3.13 so
  make verify works regardless of system python; fix make dev/deploy to
  use the workers dependency group.
- Wire audit_example_graph and three forthcoming content gates into
  quality-checks.
- Git hooks now regenerate embedded example data as well as the asset
  manifest; merge=ours covers src/example_sources_data.py too.
- verify.yml: drop duplicate PR runs, add permissions and concurrency.
- preview.yml: pass workflow inputs via env (no shell interpolation next
  to the Cloudflare token), pin wrangler, pass PBE_SMOKE_BYPASS_SECRET.
- regenerate-generated-files.yml: pull before building and retry with
  rebuild on push races, so stale generated output cannot land on main.

https://claude.ai/code/session_018EpcfENTg7herPR712vZeE
New tools:
- scripts/_common.py shares the root/loader/registry plumbing.
- check_program_covers_cells: an executable cell wholly disjoint from
  the :::program block fails; standalone_cells frontmatter opts out
  visibly.
- check_prose_duplication: verbatim repeated paragraphs, cell prose
  copying the intro, and duplicate note bullets fail.
- check_inline_links: prose links must be real /examples or /journeys
  targets; render_inline (now in src/textfmt.py) renders them as
  anchors, shared with figure captions so backticks render as code.

Gate integrity:
- score_example_criteria fails when a curated score exceeds the
  heuristic by more than --max-delta (default 1.5).
- check_quality_scores: waiver expiry must be a future ISO date, stale
  waivers flagged, score ranges validated, journey_average_min enforced.
- check_confusable_pairs defends against substring shadowing and
  accepts regex patterns.
- check_registry_integrity enforces the paired_pages relationship via
  see_also; check_no_figure_rationales enforces review_after dates;
  check_notes_supported reads every :::note block.
- audit_rubric_snapshot computes its scoreboard, waiver lines, and
  findings from live registries and labels curated judgements instead
  of hardcoding PASS verdicts; --date defaults to today.
- lint_seo_cache covers editor.js; verify_examples reports slug/filename
  mismatches cleanly; audit_example_graph --check help is accurate.

Figures:
- New while-backedge figure so the while-loops banner actually draws
  the back-edge its caption describes; loop-shape caption rewritten to
  describe the three stopping rules.
- Canvas text is XML-escaped; new well-formedness and escaping
  contracts; anchor contract validates against rendered cells (the
  walkthrough list diverges on 10 examples); reverse section-figure
  containment; rendered-page figure-presence contract; mechanical
  caption contract (5c); args-kwargs dividers computed via mono_divider.
- The marginalia gestalt now renders one card per attachment with the
  production figcaption, so caption/figure disagreements are visible in
  review; prototypes regenerate in make build and check-generated
  covers public/prototyping plus untracked fingerprint copies.

https://claude.ai/code/session_018EpcfENTg7herPR712vZeE
Factual and code corrections:
- dicts: scope the RuntimeError claim to adding/removing keys (value
  reassignment is legal).
- type-hints: teach the PEP 695 type statement; keep TypeAlias as the
  labelled-deprecated legacy spelling.
- regular-expressions: stop promising compile-time speedups the re
  module's internal cache already provides.
- string-formatting: describe 05.1f as zero-padding to width five.
- constants: doc link points at typing.Final, matching the page.
- special-methods: __lt__ alone suffices for sorted(); warn that
  hashing mutable Bags makes them unfindable after mutation; fix the
  three /data-model/* links (plus async-await's /iteration/* one) to
  real /examples/* routes.
- operator-overloading: __add__/__eq__ return NotImplemented for
  foreign types as the note instructs, demonstrated with Vector == 5.

Demonstration gaps:
- async-await shows a plain sync def (the contrast the registry pair
  demands; caught by the hardened confusable-pairs gate).
- positional-only-parameters: clamp now visibly caps 12 to 10, and a
  new cell shows scale(value=4) raising TypeError; keyword-only gets
  the same enforcement demo.
- exceptions: broken/fixed parsers now diverge visibly on a buggy call.
- operators/numbers/decorators/casts-and-any: programs now contain the
  cell code they were missing (short-circuit demo, ratio use, __doc__,
  isinstance narrowing).

Honesty and structure:
- subprocesses/threads/networking unsupported-fragments say plainly
  that Run fails in the sandbox and outputs come from real CPython
  execution at build time.
- hello-world/truthiness/unpacking cells get cell-specific prose
  instead of copied intro paragraphs; sets drops a duplicate bullet.
- match-statements notes cover mapping extra-keys; bound-and-unbound
  notes name the Py2 history; generics/paramspec notes show PEP 695
  inline syntax; bytes-and-bytearray moves to the Text section;
  comprehensions prints the set through sorted().
- async-await figure anchor follows its cell to cell-1.

Golden fixture redesigned as an explicit structural snapshot:
refresh_golden_fixture.py regenerates it and prints a reviewable
structural summary (29 changed examples this refresh); the parity gate
now compares full cell structure including kinds through the loader
with no legacy walkthrough dependence.

Worker runtime and front-end hardening:
- worker_asgi_bridge: fix the onclose handler being assigned to onopen;
  encode/decode ASGI headers as latin-1 per spec; run one lifespan per
  app instead of startup/shutdown around every request.
- main.py: constant-time smoke-bypass comparison; unknown Turnstile
  modes fail closed to requiring a challenge; POST bodies over 100 kB
  rejected with a friendly 413 page; security headers (nosniff,
  referrer-policy, frame protections) on all Worker HTML; error
  responses on cacheable paths marked no-store.
- app.py: single-pass template substitution so user code containing
  __TOKEN__ text cannot corrupt the page; escape < in embedded JSON so
  </script> in example code cannot break out of the inline script;
  honest build_dynamic_worker_code docstring.
- example.html textarea gains an accessible label; site.css adds
  --accent-text (#C83800, 4.6:1) for small accent-colored text;
  screenshot script picks the Chrome path per platform.
- New behavioral test suite for clearance signing, challenge modes,
  cookie attributes, and cache keys (15 tests); execution test now
  asserts real expected output; quality-check tests cover the new gates
  plus negative cases proving each gate can fail.

https://claude.ai/code/session_018EpcfENTg7herPR712vZeE
- example-source-format-spec: implemented status header, checklists
  ticked as the historical record, :::unsupported and the
  expected_output/standalone_cells frontmatter documented, golden
  fixture policy rewritten for the structural-snapshot design, parity
  behavior matches the rewritten gate.
- example-figure-rubric: banner ceiling corrected to the rendered-width
  formula against 640px, grammar contracts relabelled to match the test
  suite (5/5b), new 5c caption contract and Contract 11 XML
  well-formedness documented.
- visual-explainer-spec and journey-visualisation-rubric: journey
  figures documented as the centered block between heading and list
  that production ships, not the 2-column layout that never did.
- example-quality-rubric: gate wording names the enforced 9.0 target /
  8.5 hard-minimum machinery; lessons-learned notes the fixture's
  post-migration role.
- turnstile spec: unknown challenge modes documented as fail-closed.
- example-graph doc: audit script moved from future work to its
  shipped, gated reality.
- README: features cover journeys, figures, Turnstile, and the quality
  gates; architecture lists the marginalia and textfmt modules; example
  workflow includes quality-checks and the fixture refresh.
- CONTRIBUTING: full 13-script gate table, an end-to-end new-example
  workflow, and a secrets/deploy-configuration section.
- CHANGELOG: Unreleased entry covering the toolchain, gates, runtime
  hardening, and content corrections.

https://claude.ai/code/session_018EpcfENTg7herPR712vZeE
Layers the non-overlapping parts of a parallel audit-remediation pass
onto the editorial-registry/security/CSP work already in HEAD. Only the
items the follow-up commit did not cover are included here; redundant
work (confusable code-scoping, broad-tour focused_neighbors, security
headers, the score-registry move) is intentionally omitted in favour of
the existing implementation.

Content correctness:
- special-methods: __lt__ now orders by contents (consistent with __eq__,
  which the old length-based __lt__ contradicted) and the __hash__ cell
  demonstrates the mutable-hash hazard it warns about — a Bag found in a
  set becomes unfindable after its items change.
- subprocesses / threads-and-processes / networking: drop the duplicated
  :::unsupported block that restated the executable cell's code; each page
  keeps one verified cell whose prose carries the mechanism plus one
  honest sentence about the sandbox boundary.

Figures:
- Remove dead grammar vocabulary (open_arrow, dispatch, register(between=),
  the orphaned ARROW_OPEN constant) — no callers.
- FigureSizeContract gains a banner-height check (<=440px displayed at
  banner width); new FigureLineTextCollisionContract (Contract 12) rejects
  a solid line striking a label. The latter caught a real collision in
  control-stop-boundary (the first-true arrow clipped the 'd' label),
  fixed by starting the arrow below the cell row.

Docs:
- rubric-saturation: 124 figures / 7 reused (was 109 / 12).
- observability-spec: cleanup snippet gains the not-used guard.
- rubric-audit-2026-05-12: marked superseded; journey count 24 -> 21.
- figure-rubric documents Contract 12 and the height check.

https://claude.ai/code/session_018EpcfENTg7herPR712vZeE
Test-only additions covering paths the follow-up commit left at the
source-substring level:

- RunExampleFlowTests drives the real run_example POST handler through
  the verification-required (asserts the data-turnstile-required
  attribute and that code does NOT run), missing-site-key, oversize-413,
  off-mode, and verify-then-set-clearance branches. The Dynamic Worker
  run and Turnstile siteverify call are stubbed so only handler control
  flow is exercised.
- DynamicWorkerCodeTests asserts build_dynamic_worker_code embeds code via
  repr (no closing-quote breakout) and defines the entrypoint.
- Bridge process_request: stub js/pyodide.ffi/workers and drive a trivial
  ASGI app through the non-streaming path (echoes body, status, latin-1
  headers), app-exception propagation, and the new max_body_bytes 413 cap
  (asserts the app never runs on an oversize body).

https://claude.ai/code/session_018EpcfENTg7herPR712vZeE
@adewale adewale force-pushed the claude/quirky-bell-8fqrne branch from 72f383a to f6efae2 Compare July 10, 2026 02:06
@adewale adewale changed the title Audit remediation: quality gates with teeth, pinned toolchain, content and runtime fixes Audit remediation: harden runner, registry, and generated assets Jul 10, 2026
@adewale adewale changed the title Audit remediation: harden runner, registry, and generated assets Audit remediation: Worker security, content quality, and generated assets Jul 10, 2026
@adewale adewale changed the title Audit remediation: Worker security, content quality, and generated assets Audit remediation: Worker security, content, and release gates Jul 10, 2026
@adewale adewale merged commit 5a4dd0e into main Jul 10, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants